1. Field of Invention
The current invention relates generally to apparatus, systems and methods for biometrics. More particularly, the apparatus, systems and methods relate to mobile device based gait biometrics. Specifically, the apparatus, systems and methods provide for a way of using accelerometers to collect gait biometrics to identify a person.
2. Description of Related Art
Gait is the special pattern of human locomotion. It is fairly unique to an individual due to one's specific muscular-skeletal bio-mechanism. Humans can often effortlessly recognize acquaintances by the way they walk or jog. However, as a behavioral biometrics, gait may also be affected by transient factors such as tiredness, sickness, and emotions etc. In addition, external factors such as clothes, shoes, carried loads, and floor characteristics can also influence gait as well.
Automatic gait biometrics, which studies the gait using sensed data, has been an active research area receiving increasing attention over the years. Similar to fingerprints and iris biometrics, gait biometrics can be performed for two purposes: (1) identification, where a gait is compared to a database of enrolled gaits with known identities to determine whom the unknown gait belongs to, and (2) authentication, where a gait is compared to the enrolled gait data of a known person to validate the identity.
Computer vision based gait recognition extracts motion features from image sequences for gait classification. These approaches are, in general, susceptible to variations in viewing geometry, background clutter, varying appearances, uncontrolled lighting conditions, and low image resolutions. Measurements from floor pressure sensors have also been explored for gait recognition. However, these systems are usually too cumbersome to deploy for practical applications.
In the past decade, wearable inertial sensors such as accelerometers have been intensely researched for gait and activity analysis. An accelerometer measures proper acceleration and facilitates motion data collection when worn by a human body. Such sensors are advantageous compared to both videos and floor sensors for automatic gait biometrics.
While in vision based approaches, the inference of body motion from cluttered images is highly ambiguous, error prone, and vulnerable to variations in a number of external factors, accelerometers directly measure human body motion to achieve more accurate gait biometrics. Accelerometers are also inexpensive, small in size (about the size of a coin), and very easy to deploy. Mobile devices such as smart phones and tablets use accelerometers to automatically determine the screen layout for improved user experience. With the ubiquity of such devices, motion measurements of accelerations can be collected continuously and effortlessly for un-obtrusive gait-based authentication and identification, as a mere consequence of a user carrying the mobile device around.
Accelerometer based gait and activity analysis has been a popular research area since the pioneering work done by Mantyjarvi et al. about a decade ago. As is disclosed in J. Mantyjarvi, J. Himberg, and T. Seppanen, Recognizing Human Motion with Multiple Acceleration Sensors, IEEE Int'l Conf. Systems, Man, and Cybernetics, 2001 and J. Mantyjarvi, M. Lindholm, E. Vildjiounaite, S.-M. Makela, and H. Ailisto, Identifying Users of Portable Devices From Gait Pattern with Accelerometers, IEEE lnt'l Conf. Acoustics, Speech, and Signal Processing, vol. 2, pp. 973-976, 2005, the contents of which are incorporated herein. Earlier work used multiple motion sensors attached to human body parts to analyze their movements and bio kinematics. Later, data from a single sensor at a fixed position such as the feet, hips, or waist was also exploited. With the proliferation of smart phones equipped with advanced sensors, there has been a surge of research interest on the use of accelerometers in commercial off the shelf (COT) mobile devices for activity and gait classification. Unlike the dedicated sensors used in earlier research, accelerometer signals in mobile devices are usually irregularly sampled at a relatively low frame rate for power conservation and efficient resource sharing.
The most commonly used triple axis accelerometers capture accelerations along three orthogonal axes of the sensor. Given a multivariate time series of the acceleration data, feature vectors are usually extracted for signal windows corresponding to each detected gait cycle or for windows of a pre-specified size. These windows are compared and matched based on template matching, using either the correlation method or dynamic time warping. Alternatively, statistical features including mean, standard deviations, or time span between peaks in windows, histograms, entropy, higher order moments, and cumulants in spatial domain are also used. Fast fourier transforms (FFT) and wavelet coefficients in frequency domain are used to compare longer sequences. Classifiers including nearest neighbor classifier, support vector machine (SVM), and Kohonen self-organizing map have been used. In some cases, preprocessing such as weighted moving average is applied to suppress the noise in data.
Most existing research is conducted in well controlled laboratory settings: there are strict constraints on where and how the sensors are placed to reduce variation and noise in data. In some cases the sensors are placed in a specific way so that intuitive meanings can be assigned to the data components and exploited for gain analysis.
For practical world applications, it may be unrealistic to assume fixed placement of the sensor. Mobile devices are usually carried casually in pockets or hands casually without constraints in orientation. Since the same external motion results in completely different measurements with changing sensor orientation, it is essential to compute gait biometrics robust to sensor rotation for realistic scenarios. However, research on this aspect is rather scarce. Mantyjarvi et al. used both principle component analysis (PCA) and independent component analysis (ICA) to discover “interesting directions” to compute gait features for activity analysis. The underlying assumption of identical data distributions for both training and testing data are unlikely to hold for realistic applications and computed gait features based on magnitude measurements. However, the computation of the uni-variate magnitude series using the raw 3D multivariate series resulted in information loss and ambiguity artifacts.
One approach to this challenge is augmenting the training set with simulated data at multiple sensor orientations by artificially rotating available training data. However, significant artificial sampling needed to tessellate the 3D rotational space creates unbearable computational and storage burden with the additional risk of degraded classifier performance. Orientation invariant features were extracted using the power spectrum of the time series. However, it suffered shortcomings common to frequency domain methods: loss of temporal locality and precision, and vulnerability to drifting in gait tempo. Others have used a co-built-in gyroscope sensor to calibrate accelerometer data to the up-right posture in order to reduce the influence of noise in sensor orientation. This approach requires calibration prior to every data collection, expects the sensor to not rotate during data collection, only relieves noise in the vertical direction, and makes unrealistic assumptions that all poses are up-right.
These studies paint a picture of drastic degradation in gait recognition performance in the more relaxed scenarios. Even with the new invariant features, accuracy at approximately 50% was reported. On the other hand, performances in the high 90s are often achieved in more controlled scenarios. Although each study uses its own dataset and evaluation standards so the numbers are not directly comparable, the constant large gap in performance does highlight the challenge in realistic gait biometrics using orientation dependent motion sensors.
Although state-of-the-art accelerometer based gait recognition techniques work fairly well under constrained conditions, their performance degrades significantly for real world applications where variations in sensor placement, footwear, outfit, and performed activities persist. For a mobile device based gait biometrics system to succeed, it is crucial to address the variations in sensor orientation due to casual handling of mobile devices.
Despite a surge in research efforts, accelerometer based gait biometrics remains a challenge for practical applications due to data dependency on sensor placement: accelerations are measured along the sensor axis. The measurements change with sensor orientation even when body motion stays the same. Most existing research is conducted in fixed laboratory settings with restricted sensor placement to bypass this problem, and is vulnerable in real world usage where the placement of mobile devices is casual and even arbitrary. Although promising performance is reported in well-controlled studies on gait biometrics using accelerometers, there still is a large gap between existing research and real world applications.
A better way of identifying a person's gait is desired.